Chinese Named Entity Identification Using Class-based Language Model1

We consider here the problem of Chinese named entity (NE) identification using statistical language model(LM). In this research, word segmentation and NE identification have been integrated into a unified framework that consists of several class-based language models. We also adopt a hierarchical structure for one of the LMs so that the nested entities in organization names can be identified. The evaluation on a large test set shows consistent improvements. Our experiments further demonstrate the improvement after seamlessly integrating with linguistic heuristic information, cache-based model and NE abbreviation identification. ,QWURGXFWLRQ 1( LGHQWLILFDWLRQ is the key technique in many applications such as information extraction, question answering, machine translation and so on. English NE identification has achieved a great success. However, for Chinese, NE identification is very different. There is no space to mark the word boundary and no standard definition of words in Chinese. The Chinese NE identification and word segmentation are interactional in nature. This paper presents a unified approach that integrates these two steps together using a class-based LM, and apply Viterbi search to select the global optimal solution. The class-based LM consists of two sub-models, namely the context model and the entity model. The context model estimates the probability of generating a NE given a certain context, and the entity model estimates the probability of a sequence of Chinese characters given a certain kind of NE. In this study, we are interested in three kinds of Chinese NE that are most commonly used, namely person name (PER), location name (LOC) and organization name (ORG). We have also adopted a variety of approaches to improving the LM. In addition, a hierarchical structure for organization LM is employed so that the nested PER, LOC in ORG can be identified. The evaluation is conducted on a large test set in which NEs have been manually tagged. The experiment result shows consistent improvements over existing methods. Our experiments further demonstrate the improvement after integrating with linguistic heuristic information, cache-based model and NE abbreviation identification. The precision of PER, LOC, ORG on the test set is 79.86%, 80.88%, 76.63%, respectively; and the recall is 87.29%, 82.46%, 56.54%, respectively. 5HODWHG :RUN